1. Field of the Invention
The present invention relates generally to methods for performing pen-based computing, and in particular, to a method and apparatus for storing and manipulating ink stroke representations. Still more particularly, the present invention relates to a note taking system that provides simplified user manipulation of handwritten ink strokes to manage space, improved paragraph parsing with specialized character recognition and new methods for storing and displaying ink stroke representations.
2. Description of Related Art
Pen-based computers in which the pen replaces the keyboard and/or mouse as the primary input device have become well known. Such systems provide a digitizing pad and pen for the user to input information. The information is input in the form of strokes or graphic representations of strokes. The computer monitors the movement of the pen and creates a stroke when the pen comes within a pre-defined proximity of or in contact with the digitizing pad. The computer follows the movement of the pen across the digitizing pad while simultaneously extending the stroke, and ends the stroke when the pen is removed or moved out of proximity from the digitizing pad.
While programs have been created for converting pen-based input into characters, they are limited because they have problems dividing the input where the user enters several strokes just as if he/she were writing upon a blank sheet of paper. One particular problem is the recognition of dashes and bullets commonly used in lists and outlines. Particularly, bullet characters filled in with multiple strokes are rarely recognized correctly. These characters are not only difficult to recognize, but existing recognition systems require an inordinate amount of time to perform such recognition. For example, there are currently no character recognition systems that can recognize characters xe2x80x9con the flyxe2x80x9d as the user is inputting the ink strokes into the system. Thus, the user is forced to wait while the system processes a stroke input, before additional strokes can be input. Therefore, there is need for a system and methods for quickly and easily recognizing ink strokes such as bullets and dashes, or performing recognition in the background and managing the processing resources.
One of the major problems for users of pen-based systems is managing space. In most pen-based computers, there is no grouping of the ink strokes by paragraph, line or word to allow the user to perform standard word-processing fimctions. Most computers do not separate ink strokes into words because the spacing between characters can vary greatly depending on the writing style of the person inputting the data. Moreover, the pen is typically the only input device in pen-based systems, and space cannot be input by selecting a space or return key. Thus, it is difficult to manage the space between words on the same line or the number of lines between words on different lines. Therefore, there is a need for a means that allows the user to insert and remove space in a pen-based computing system.
Most prior art systems have attempted to resolve this problem by providing dedicated gestures, or two step gesture methods that add space or remove space from between electronic representations of strokes. However, such prior art approaches have not been successful at resolving the space problem for a variety of reasons. One reason is that the gestures are not easily discoverable. In other words, the user must memorize gestures or specific manipulations of the pen or input device that are not well known and not intuitive as to the function that will be performed. Another reason is that gestures are difficult for the user to input; they are complex gestures that require a very specific manipulation of the input device. The input device must often be manipulated several times before the gesture is correctly input. Yet another problem with existing space gestures is that the inputs are not always correctly recognized as space management gestures. When the gestures are not correctly recognized, they are interpreted as ink strokes which forces the user to perform even more steps to remove the ink strokes. Moreover, it is particularly difficult with gestures to determine the point between which ink strokes where space will be added or removed. Thus, there in a need for a system and method for adding and removing space in pen-based computer system that is easy to use.
Another problem with existing computing systems is the ability to interchangeably use different types or modes of operation such as word-processing functionality and drawing/graphic functionality. Particularly in programs that provide a seamless transition between application programs or modes of operation in a single application program, it is difficult for the user to identify the mode in which he or she is operating. This is problematic because each mode of operation provides commands and may respond differently to the same inputs. For example, a particular command or gesture may perform a first action if the user is operating on a drawing element or item and perform a second, completely unrelated and different action if the user is operating on a word processing element or item. Notifying the user of the commands available to manipulate a stroke is particularly problematic because it is often very difficult for the user to determine whether a stroke is associated with word-processing functionality or drawing/graphic functionality. Therefore, there is need for a system that provides the user with feedback as to the current mode of operation.
Yet another shortcoming of existing pen-based computer systems, is that they suffer from major performance problems for storage, display and printing of ink stroke representations. As the user manipulates the input device over the digitizing pad, a great number of points are generated as the input data forming a single stroke. The amount of memory required to store the data forming several strokes is very large, therefore, pen-based systems require significant amounts of random access memory (RAM) and disk space. The processing of the input data is also very computationally intensive because of the large number of points forming each stroke. Thus, the processing of the ink stroke representations to perform character recognition, to display the strokes on the display device, or even to print the strokes on a hard copy device requires significant amounts of time. Therefore, there is a need for a system and method that can reduce the amount of time required to display, print or perform character recognition on ink stroke representations, as well as reduce the storage requirements for the user.
The present invention overcomes the limitations and shortcomings of the prior art with an improved system and method for spacing, storing and recognizing electronic representations of handwriting, printing, computer text, bit maps, and drawings. The system of the present invention comprises a central processing unit that couples with a display device, an input device, and a memory. The memory includes means for spacing ink stroke representations, means for recognizing outline elements, means for performing deferred recognition, means for highlighting drawing areas and means for storing and displaying ink stroke representations. These means operate together to provide a pen-based computer system with improved performance and added functionality for manipulating the space between ink stroke representations. The present invention is particularly advantageous because the means for spacing ink stroke representations provides the user with a tool that can be easily manipulated and that notifies the user as to the amount of space that will be added or removed.
The preferred embodiment of the present invention comprises a plurality of novel methods that provide this functionality such as: a method for managing space between ink stroke representations, a method for displaying drawing elements in a visually distinct manner, a method for recognizing bullets and dashes, a method for storing ink stroke representations, a method for displaying and printing ink stroke representations, and a method for performing deferred character or word recognition. The methods of the present invention are particularly advantageous because the methods for storing, printing and displaying ink stroke representations greatly improves overall system performance by reducing the amount of memory required to store ink stroke representations, and by reducing the amount of time required to render ink stroke representations on a display device or on a printer.